United States Patent and Trademark Office 



UNITED STATES DEPARTMENT OF COMMERCE 
United States Patent and Trademark Office 
Address: COMMISSIONER FOR PATENTS 
P.O. Box 1450 

Alexandria, Virginia 22313-1450 
www.uspto.gov 



APPLICATION NO. 



FILING DATE 



FIRST NAMED INVENTOR 



ATTORNEY DOCKET NO. 



CONFIRMATION NO. 



10/064,873 



44702 



08/26/2002 



Gopal B. Avinash 



7590 08/02/2006 

OSTRAGER CHONG FLAHERTY & BROITMAN PC 
250 PARK AVENUE, SUITE 825 
NEW YORK, NY 10177 



15-UL-6174 



6172 



EXAMINER 



SETH, MANAV 



ART UNIT 



PAPER NUMBER 



2624 

DATE MAILED: 08/02/2006 



Please find below and/or attached an Office communication concerning this application or proceeding. 



PTO-90C (Rev. 10/03) 



Office Action Summary 


Application No. 

10/064,873 


Applicant(s) 
AVI NASH ET AL. 


Examiner 

Manav Seth 


Art Unit 

2624 





The MAILING DATE of this communication appears on the cover sheet with the correspondence address 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

Responsive to communication(s) filed on 10 July 2006 . 
2a)D This action is FINAL. 2b)S This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) KI Claim(s) 1-32 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) ^ Claim(s) 1-32 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) D The drawing(s) filed on is/are: a)Q accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

1 1) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)D All b)D Some * c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2. Q Certified copies of the priority documents have been received in Application No. . 

3. Q Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



Attachment(s) 

1 ) □ Notice of References Cited (PTO-892) 

2) O Notice of Draftsperson's Patent Drawing Review (PTO-948) 

3) □ Information Disclosure Statement(s) (PTO-1449 or PTO/SB/08) 

Paper No(s)/Mail Date . 



4) □ Interview Summary (PTO-413) 

Paper No(s)/Mail Date. . 

5) [U Notice of Informal Patent Application (PTO-152) 

6) □ Other: . 



U.S. Patent and Trademark Office 

PTOL-326 (Rev. 7-05) 



Office Action Summary 



Part of Paper No./Mail Date 20060726 



Application/Control Number: 10/064,873 Page 2 

Art Unit: 2624 

DETAILED ACTION 
Continued Examination Under 37 CFR 1.114 

1. A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 
CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for 
continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1 .17(e) has been timely 
paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. 
Applicant's submission filed on July 10, 2006 has been entered. 

Response to Amendment 

2. Applicant's amendment filed on July 10, 2006 has been considered and entered in full. 

3. Applicant's amendments with respect to respective claims have been considered but are 
moot in view of the rejection(s) made below. 

Claim Refections - 35 USC § 112 

4. The following is a quotation of the first paragraph of 35 U.S.C. 112: 

The specification shall contain a written description of the invention, and of the manner and process of making and 
using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or 
with which it is most nearly connected, to make and use the same and shall set forth the best mode contemplated by 
the inventor of carrying out his invention. 
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5. Claims 1-32 are rejected under 35 U.S.C. 112, first paragraph, as failing to comply with the 
written description requirement. The claim(s) contains subject matter, which was not described in 
the specification in such a way as to reasonably convey to one skilled in the relevant art that the 
inventor(s), at the time the application was filed, had possession of the claimed invention. 

The amendment filed July 10, 2006 presents the amendment to the claims, which recites 
"ultrasound image". The claims and specification defining applicant's invention as originally filed 
does not provide any support for the word "ultrasound" associated with the image. However, 
examiner does acknowledge the section "background of invention" which recites "In many 
applications, such as medical diagnostic imaging, images are saved with annotations burnt in" (para. 
0002) and further provides an example of ultrasound imaging (para. 0003) but no support of further 
particularly processing the ultrasound images has been found in the claims and specification defining 
applicant's invention as originally filed. The applicant or inventor of the instant application, at the 
time the application was filed, had no possession of the invention as cited above in the section 
"background of invention' and further does not show any support of processing the ultrasound 
images in the specification but rather broadly recites processing the annotated image and processing 
the annotated image has been rejected before in the previous office action. 



Claim Rejections - 35 USC § 103 



6. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness 
rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 
102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the 
subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary 
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skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the 
invention was made. 

7. Claims 1,13 and 30 are rejected under 35 U.S.C. 103(a) as being unpatentable over Ikeshoji 
et al., U.S. Patent No. 5,761,339, further in view of applicant's admitted prior art (here AAA). 

Regarding claims 1, 13 and 30, Ikeshoji discloses removing one or more textual 
annotations form a grayscale annotated image to derive a first modified image (figure 1; element 
20D being the first modified image, which is obtained after annotations (elements 30, 32 and 34) are 
removed; element 30D represents removed annotations only with no background image where 
annotations being textual (characters) (See Title; figure 1 clearly says image 30D being character and 
figure image; col. 2, lines 36-42 discloses characters (textual annotations) are drawn on an image). 

Ikeshoji further discloses processing said first modified image (20D) using an algorithm to 
derive a processed image (col. 5, lines 1-5; removing stains). 

Ikeshoji further discloses merging the removed one or more annotations with said processed 
image to derive a merged image (figure 3, element 10-1 being the merged image). 

Ikeshoji further teaches "The present invention relates to an image processing method for 
correction of an image when a natural image such as a photograph or a picture or a document of 
characters and/or figures drawn on paper is read by an electronic means such as a scanner and 
recorded on a recording medium or displayed on a display" (col. 1, lines 8-13). Clearly from the all 
of the above disclosure described, Ikeshoji describes a scanner to obtain an image and then records 
the images electronically on recording medium and then performs image processing functions on 
the images with annotations on it as described above but does not does not expressly teach using 
these image processing functions of an ultrasound image. The question now is that can Ikeshoji's 
image processing functions be used on images scanned by ultrasound device? The answer would be 
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yes because it is a well known fact that any image when stored electronically is prone to any kind of 
image processing that can be performed electronically and the ultrasound image is just an image like 
any other normal image when stored electronically and as well known in the art of imaging, 
ultrasound device is nothing but a scanner that obtains images and then records these images in 
electronically in the memory and further these images are easily accessible to the operator or user for 
further processing or displaying and Applicant's admitted prior art further teaches such a medical 
diagnostic imaging, particularly ultrasound imaging and further teaches saving ultrasound images 
into the memory and then performing image processing functions to improve the image on which 
the annotation is burnt in (specification, para 0002-0003), as similarly done by Ikeshoji. Therefore, it 
would have been obvious for one of ordinary skill in the art at the time of invention was made to 
combine the teachings of Ikeshoji and applicant's admitted prior art (AAA) because both references 
are directed to the processing of annotated images and solves the same problem of improving 
annotated images and Ikeshoji's provides image processing algorithms that can be applied to 
ultrasound images of AAA since an ultrasound image is just an image like any other image stored 
electronically and all the images being defined by pixels in electronic format, the same image 
processing which Ikeshoji applies to the image pixels can be applied to the ultrasound images stored 
electronically. 

8. Claims 2-12, 14-21 and 31-32 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Ikeshoji et al., U.S. Patent No. 5,761,339, further in view of applicant's admitted prior art, and 
further in view of Macleod et al., U.S. Patent No. 5,778,092, and further in view of Bloomberg, U.S. 
Patent No. 5,065,437. 
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Claim 2 recites "The method as recited in claim 1, wherein said removing step comprises 
the following: deriving a first binary mask defining one or more image regions; and multiplying said 
first binary mask and said annotated ultrasound image to derive said modified image". Ikeshoji 
clearly teaches in figure 1, deriving a first modified image (element 20 D) from which annotations 
have been removed with the help of filtering (or masking). Removing a part of images by using a 
mask is very well known in the process of image segmentation where processing such as 
convolution further involving multiplying mask bits with original image is used and is generally used 
in differentiating and removing an image part from the original image. Ikeshoji does teach of using a 
filter or mask but does not specifically teach deriving a binary mask, however, Macleod teaches this 
generally used well-known method of deriving a binary mask (col. 2, lines 1-13; col. 4, lines 65-68 
through col. 5, lines 1-16 and lines 30-68; col. 8, lines 4-10 and lines 25-40). 

Further supporting, in lines 57-62 of col. 11, MacLeod discloses that U.S. Patent No. 5,065,437, is 
incorporated by reference, thereby, not requiring the motivation to combine the references. 
Bloomberg. Bloomberg provides the support for subject matter in claim 2 in (figure lB,col. 2, lines 
20-68; col. 4, lines 1-68; col. 6, lines 10-40). Therefore, it would have been obvious for one of 
ordinary skill in the art at the time of invention was made to use combined teachings of MacLeod 
and Bloomberg in the invention combined of Ikeshoji and AAA because all references are directed 
to image segmentation and the combined MacLeod and Bloomberg's teachings provide the well- 
known teachings that are generally used in image segmentation, which would further provide a 
robust and computationally efficient technique for identifying and separating regions (See 
Bloomberg, col. 2, lines 5-15). 
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Claim 3 recites "The method as recited in claim 2, wherein said merging step comprises the 
following" inverting said first binary mask to derive a second binary mask defining one or more 
annotation regions; multiplying said second binary mask and annotated ultrasound image to derive a 
modified image; and merging said modified image and said processed image to derive said merged 
ultrasound image". Ikeshoji clearly shows in figure 1, deriving images 20D and 30D, where in image 
20D annotations are removed and in image 30D only annotations are present, therefore these 2 
images are clearly subject to the apparent use of a mask that when inverted would provide 
segmentation for extracting one part or the other from the original image. Figure 3 clearly shows 
merging said second modified image (30) and said processed image (20) to derive said merged image. 

Claim 4 recites "The method as recited in claim 1, wherein the merged textual annotations 
occupy the same pixels in said merged ultrasound image that the removed textual annotations 
originally occupied in said annotated ultrasound image". Ikeshoji clearly shows in figure 3 the image 
10-1 which has annotation at the same image position as shown in figure 1 image 10 and further 
support can be found in (col. 4, lines 6-12). 

Claim 5 recites "the method as recited in claim 1, wherein said removing step comprises 
morphology-based processing and thresholding". MacLeod discloses dilation and erosion 
(morphological operations) (col. 7) and further teaches thresholding (col. 5). 

Claim 6 recites "The method as recited in claim 1, wherein said removing step comprising 
the following: grayscale erosion of said annotated ultrasound image using a structuring element to 
derive an eroded image; thresholding said eroded image to derive a first binary mask; dilation of said 
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first binary mask using said structuring element to derive a second binary mask defining one or more 
image regions; and multiplying said second binary mask and said annotated ultrasound image to 
derive said modified ultrasound image". The subject matter recited in the claim 6 is nothing but a 
morphological operation (opening: erosion followed by dilation), which is very well-known to be 
used in the process of identifying and separating image portions. MacLeod clearly teaches all the 
steps recited in claim 6 in (col. 11 through col. 12, lines 1-55). Bloomberg further provides the 
support in (col. 9, lines 45-68 through col. 10, lines 1-25). 

Regarding claim 7, citing the well-known fact of modifying the image using an image mask, 
and in further view of Ikeshoji providing merging of images as explained before in the rejection of 
claim 3. Claim 7 has been similarly analyzed and rejected as per claims 3 and 6. 

Regarding claim 8, MacLeod discloses said removing step comprises thresholding and 
pixel connectivity-based analysis" (col. 6, lines 44-56). See Bloomberg (col. 2, lines 28-60; col. 4, lines 
40-65; col. 7, lines 1-65). 

Regarding claim 9, claim 9 additionally recites the limitation where "using 8-connected 
analysis to reject segments smaller than a prespecified size from said first binary mask to 
derive a second binary mask". MacLeod discloses of the same in (col. 6, lines 44-56; col. 10, lines 1- 
40). All other limitations have been similarly analyzed and rejected as per claims 1-8. 

Regarding claim 10, claim 10 has been similarly analyzed and rejected as per claims 9, 7 and 

3. 
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Regarding claim 11, claim 1 1 in addition to claim 9 recites "removing holes from a second 
binary mask to derive a third binary mask". MacLeod teaches mask reconstruction for filling 
(removing) the interior holes (col. 11, lines 63-68 through col. 12, lines 1-15). 

Regarding claim 12, Ikeshoji discloses filtering to enhance said modified ultrasound image 
(col. 6, lines 10-20). 

Claims 14-21 have been similarly analyzed and rejected as per claims 2-12. 

Claim 31 and 32 have been similarly analyzed and rejected as per claims 2-3. 

9. Claims 22-29 are rejected under 35 U.S.C. 103(a) as being unpatentable over Ikeshoji et al, 
U.S. Patent No. 5,761,339, further in view of applicant's admitted prior art, and further in view of 
Macleod et al, U.S. Patent No. 5,778,092, and further in view of Bloomberg, U.S. Patent No. 
5,065,437 and further in view of Gonzales et al., 1992, Book publication "Digital image processing". 

As discussed before in the rejection of claims 1-12, the combined invention of Ikeshoji, 
AAA, MacLeod and Bloomberg provides techniques that are used on grayscale images. Now the 
question is, can the same techniques be used on color images? The answer would be yes and further 
support is provided by Gonzales. Gonzales teaches "the HIS model is ideally suited for image 
enhancement, because the intensity component is decoupled from the color information in an 
image. Therefore any monochrome enhancement technique can be carried over as a tool for 
enhancing full-color images. It simply calls for converting the image to the HIS format, processing 
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the intensity component, and converting the result to RGB for display" (page 247, 4 th paragraph) 
and apparendy the HS components would have to be merged back in as the processing is done on 
color images and the output required would be a color image. Also, converting an RGB to HSI and 
HSI to RGB model is very well known and is further taught by Gonzales on page 229 and 235. 
Therefore, providing a motivation for one of ordinary skill in the art at the time of invention was 
made to combine Gonzales's teachings in the combined invention of Ikeshoji, AAA, MacLeod and 
Bloomberg. All other limitations of claims 22 and 26 have been similarly analyzed and rejected as 
per claims 1-12. 

Claims 23-24 and 27-28 have been similarly analyzed and rejected as per claims 2-3, 22 and 

26. 

Claims 25 and 29 have been similarly analyzed and rejected as per claims 22 and 26. 



Any inquiry concerning this communication or earlier communications from the examiner 
should be directed to Manav Seth whose telephone number is (571) 272-7456. The examiner can 
normally be reached on Monday to Friday from 8:30 am to 5:00 pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, 
Bhavesh Mehta, can be reached on (571) 272-7453. The fax phone number for the organization 
where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR system, 
see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, 
contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 
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